Multilayer Perceptron Learning Utilizing Singular Regions and Search Pruning
نویسندگان
چکیده
In a search space of a multilayer perceptron having J hidden units, MLP(J), there exist flat areas called singular regions. Since singular regions cause serious stagnation of learning, a learning method to avoid them was once proposed, but was not guaranteed to find excellent solutions. Recently, SSF1.2 was proposed which utilizes singular regions to stably and successively find excellent solutions commensurate with MLP(J). However, SSF1.2 has a problem that it takes longer as J gets larger. This paper proposes a learning method SSF1.3 that enhances SSF1.2 by attaching search pruning so as to discard a search whose route is similar to one of previous searches. Our experiments showed SSF1.3 ran several times faster than SSF1.2 without degrading solution quality.
منابع مشابه
Fukumizu : Statistical Active Learning in Multilayer Perceptron 3
|This paper proposes new methods of generating input locations actively in gathering training data, aiming at solving problems special to multilayer perceptrons. One of the problems is that the optimum input locations which are calculated deterministically sometimes result in badly-distributed data and cause local minima in back-propagation training. Two probabilistic active learning methods, w...
متن کاملNeurodynamics of Learning and Its Effect on Network Performance
Using a simple dynamic model, we show that performance equal to or better than the Probabilistic Neural Netwok (PNN) can be achieved with a single three-layer Multilayer Perceptron (1fLP), by making fundamental changes in the network optimization strategy. These changes are: 1) Neuron activation functions are used which reduce the probability of singular Jacobians; 2) Successive regularization ...
متن کاملActive Learning in Multilayer Perceptrons
We propose an active learning method with hidden-unit reduction, which is devised specially for multilayer perceptrons (MLP). First, we review our active learning method, and point out that many Fisher-information-based methods applied to MLP have a critical problem: the information matrix may be singular. To solve this problem, we derive the singularity condition of an information matrix, and ...
متن کاملAn Efficient Method for Pruning the Multilayer Perceptron Based on the Correlation of Errors
In this paper we present a novel method for pruning redundant weights of a trained multilayer Perceptron (MLP). The proposed method is based on the correlation analysis of the errors produced by the output neurons and the backpropagated errors associated with the hidden neurons. Repeated applications of it leads eventually to the complete elimination of all connections of a neuron. Simulations ...
متن کاملKnowledge-based fuzzy MLP for classification and rule generation
A new scheme of knowledge-based classification and rule generation using a fuzzy multilayer perceptron (MLP) is proposed. Knowledge collected from a data set is initially encoded among the connection weights in terms of class a priori probabilities. This encoding also includes incorporation of hidden nodes corresponding to both the pattern classes and their complementary regions. The network ar...
متن کامل